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ABSTRACT 


The  entropy  formulas  of  information  theory  are  not  rele¬ 
vant  to  the  cost  of  obtaining  knowledge  nor  to  its  value  in 
use.  But  they  are  relevant  to  the  expected  number  of  symbols 
needed  for,  and  hence  to  the  cost  of^ storing  and  transmitting 
messages,  especially  in  long  sequences.  This  is  analogous  to 
the  cost  of  storing  and  transporting  material  commodities,  de¬ 
pending  as  it  does  on  their  weight  or  volume;  while  a  commo¬ 
dity*  s  production  cost,  and  its  value  in  use,  depend  on  other 
factors . 

It  is  also  worth  recalling  that  to  increase  the  "order” 
of  an  aggregate  is  to  decrease  the  number  of  its  distinct 
possible  states.  This  decreases  the  expected  number  of  symbol 
needed  to  identify  a  state,  and  thus  decreases  the  entropy. 


1 .  Communication  Economics 


Tho  expected  cost s  and  delays  involved  in  communicating 
(i.e.,  storing,  coding  and  transmitting)  information  must  be 
carefully  distinguished  from  the  costs  and  delays  in  producing 
information,  i.e.,  in  collecting  data  about  the  environment 
by  sampling,  experimenting,  etc. 

Moreover,  both  kinds  of  expected  costas  just  mentioned 
must  be  distinguished  from  the  expected  benefits  ("information 
value:  the  negative  of  the  "Bayesian  risk")  accruing  to  the 
decision-maker  who  chooses  his  actions  on  the  basis  of  mess¬ 
ages  received.  (l-Je  assume  'utility"  to  be  separable  into 
benefit  and  cost) . 

Finally,  the  delays  in  producing  and  communicating  in¬ 
formation  cause  two  kinds  of  distinct  detriments  to  the  ex¬ 
pected  benefit:  on  the  one  hand,  the  delayed  benefit  is  dis¬ 
counted  over  time;  on  the  other  (especially  in  the  case  of  a 
"Markovian"  environment) ,  expected  benefit  may  be  diminished 
when  decisions  are  based  on  obsolete  messages. 

In  the  case  of  commodities,  the  distinction  between  pro¬ 
duction  costs  and  the  costs  of  storage  and  transportation, 
and  the  distinction  between  these  costs  and  the  value  in 
use,  is,  of  course,  familiar  to  economists  and  managers. 

The  physical  volume  or  weight  of  goods  in  warehouses  and  in 
transit  is,  by  and  large,  the  main  determinant  of  the  storage 
and  transportation  cost.  But  the  cost  of  producing  and  the 
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value  of  using  one  unit  of  weight  or  volume  widely  differ 
from  commodity  to  commodity. 

Correspondingly,  the  number  of  symbols  needed,  on  the 
average,  to  store  and  transmit  a  message  does,  by  and  large, 
determine  the  cost  of  communication,  regardless  of  how  costly 
was  the  provision  of  data,  and  how  beneficial  (i.e.,  impor¬ 
tant)  to  the  decision-maker  are  the  messages  he  receives. 

This  average  number  of  symbols  is  given  by  the  entropy 
removed  ("information  amount  transmitted"),  a  parameter  of 
the  joint  probability  distribution  of  the  inputs  and  outputs 
of  a  communication  system. 

During  the  last  decade,  the  economic  role  of  the  entropy 
parameter,  and  its  insufficiency  as  a  measure  of  information 
value,  has  been  recognized,  or  partly  recognized,  by  authors 
in  the  fields  of  engineering,  logic,  and  statistics:  see, 
for  example,  Howard  (1966),  Carnap  (1966),  DeGroot  (1970). 
However,  many  workers  in  the  field,  fascinated  by  the  simple 
properties  of  the  entropy  formula  as  the  measure  of  informa¬ 
tion  amount,"  have  tried,  rather  recently,  to  use  it  for 
comparisons  of  the  expected  benefits  of  alternative  informa¬ 
tion  systems.  In  U.S.  as  well  as  in  U.S.SR.  [Bongard 
(1963)]  and  C.S.R.  [Perez  (1967)],  attempts  have  been  made 
to  specify  the  benefit  function  (of  action  and  environment) 
in  such  a  manner  as  tc  make  the  expected  benefit  from  inform¬ 
ation  depend  on  entropy.  How  far  this  approaches  economic 


reality,  is  worth  investigating. 

!-’e  stated  above  that  the  cost  of  communication,  just 
like  the  cost  of  storing  and  transporting  commodities,  is, 
by  and  large .  independent  of  the  cost  of  production  and  the 
value  in  use.  This  is  only  an  approximation,  ihe  more 
valuable  a  commodity  the  more  will  the  profit  be  affected 
by  breakage  or  leakage,  per  pound  or  gallon.  Similarly, 
some  distortions  of  messages  reduce  the  benefit  u^re  than 
others.  As  suggested  by  Shannon  (1960)  and  further  explored 
by  Jellinek  (1963) ,  ?ham-Huu-Tri  (1968)  and  others,  the  code 
must  be  adjusted  to  a  "fidelity  criterion,"  which  is,  in 
fact,  one  aspect  of  the  benefit  function. 

Entropy  formulas  (in  "bits"  of  information)  approximate 
the  expected  number  of  necessary  symbols,  presupposing 
efficient  encoding  of  events  grouped  into  long  sub-sequences 
("blocks") .  This  is  economically  relevant  only  if  events 
follow  in  quick  succession,  permitting  to  neglect  the  delays 
and  storage  costs  caused  by  v/aiting  for  the  completion  of  each 
block.  Therefore, many  real  communication  problems  will  re¬ 
quire  a  cost  analysis  of  alternative  codes  based  on  relatively 
short  "blocks"  ;  and  ■  channels  with  equal  capacities  may 
call  for  different  codes. 


i.e.,  the  number  of  bits 


The  "transmission  rate,” 
transmitted,  per  unit  of  time,  through  a  given  communication 
channel  is  a  peiram-eter  of  the  joint  probability  distribution 
of  the  channel’s  inputs  and  outputs.  H.  Theil  (1967)  has 
pointed  out  that  since  the  relative  shares  of  the  components 
of  a  balance  sheet,  or  of  the  national  product,  or  of  any 
other  total,  are  ncn-negative  and  add  up  to  unity,  they  can 
be  treated  as  probabilities.  Accordingly,  he  suggested  to 
use  the  formula  of  the  transmission  rate  as  a  measure  of  dis¬ 
crepancy  between  the  predicted  and  the  actual  relative  shares 
of  any  composite  total.  Such  a  measure  can  be  used,  of 
course,  outside  of  economics  as  well;  and  other  measures 
commonly  used  to  express  the  goodness  of  prediction  might  be 
applied.  The  comparative  economic  usefulness  of  any  of  those 
measures  would  depend  on  their  role  in  guiding  decisions. 

The  suggested  measure  is  related  to  information  economics 
only  in  a  purely  formal  way:  viz.,  insofar  as  the  same  for¬ 
mula  is  indeed  useful  for  decisions  about  the  choice  of 
communication  channels. 

N.  Georgescu-Roegen  (1966)  has  given  good  reasons  for 
an  interesting  suggestion:  to  characterize  the  degree  of 
economic  or  social  order  by  the  negative  of  entropy.  This 
is  analogous  to  a  similar  characterization  of  living  organisms 
[e.g.,  by  Schroedinger  (1944)].  I  should  like  to  note  that. 
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again,  such  a  degree  of  order  does  correspond,  presumably, 
to  the  number  of  symbols  needed  to  describe  the  state  of  an 
aggregate  such  as  a  society  or  an  organism,  or  a  random 
collection  of  particles.  The  planner,  the  manager,  the 
"sorter^  the  "filter"  performs  a  many-to-one  mapping  or, 
equivalently,  a  partitioning  (into  two  equivalence  classes 
in  the  case  of  "Maxwell's  demon").  The  number  of  possible 
distinct  states  and  hence  the  number  of  symbols  needed  to 
identify  them  is  thus  diminished. 

2 .  Information  Systems  as  Chains  and  Networks 

The  analysis  of  expected  benefits  and  costs  of  informa¬ 
tion,  with  due  regard  to  transmission  fidelity  and  the  cost 
of  communication  was  carried  out  in  Marschak  (1968,  1970)  by 
considering  a  system  of  processors,  beginning  with  the  pro¬ 
cessing  of  events  .into  data,  and  ending  with  the  processing 
of  decoded  messages  into  decisions.  Each  processor  is 
characterized  by  a  transition  matrix,  and  by  a  cost  function 
and  a  delay  function  of  the  input.  The  assumption  (usual  in 
statistical  decision  theory)  of  costless  and  in  ;tantaneous 
decision  making  need  not  ve  mnde.  The  user  chooses  the 
system  maximizing  the  expected  difference  between  benefit  (a 
function  of  the  sequence  of  events  and  decisions)  and  total 
cost  —  provided  utility  is  additive  in  these  two  components. 


The  system  considered  so  far  has  been  a  chain :  each 
processor  receives  its  input  from  a  single  predecessor  and 
sends  its  out pi ts  to  a  single  successor.  The  following 
links  of  the  chain  were  considered:  inquiring,  storing, 
encoding,  transmitting,  decoding,  deciding. 

More  generally,  a  network  rather  than  a  chain  of  inform¬ 
ation  processors  is  used  in  practice.  Each  processor  is  a 
node,  it  receives  its  inputs  from  and  sends  its  outputs  to 
several  other  nodco.  iia  before,  the  initial  inputs  are 
given  by  the  "environment"  and  the  terminal  outputs  consti¬ 
tute  the  "action",  but  more  than  one  node  can  receive  the 
environmental  inputs,  and  more  than  one  node  represents  de¬ 
ciding  about  the  (multidimensional)  action  of  the  system. 

In  particular,  an  organization  as  viewed  by  an  "organizer" 
(characterized  by  an  objective  or  "utility"  function)  can  be 
regarded  as  an  information  network.  This  approach  was  taken 
in  Chapter  8  of  Marschak  and  Radner  (1970) .  The  approach 
should  be  modified  in  various  ways:  by  considering  the  pro¬ 
cessors  as  stochastic  transformations  (instead  of  using  an 
additive  "noise"  input) ;  by  explicitly  introducing,  for  each 
processor,  a  cost  and  delay  function  of  its  inputs;  and  by 
regarding  "incentives"  as  costs. 

On  the  other  hand,  there  is  a  clear  relation  to  the 
mathematical  theory  of  network  flows  —  for  example,  iri 
(1969)  —  provided  the  delays,  costs  and  benefits  are  intro¬ 
duced  explicitly. 
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The  entropy  formulas  of  information  theory  are  not  relevant  to  the 
cost  of  obtaining  knowledge  nor  to  its  value  in  use.  But  they  are 
relevant  to  the  expected  number  of  symbols  needed  for,  and  hence  to 
the  cost  of,  storing  and  transmitting  messages,  especially  in  long 
sequences.  This  is  analogous  to  the  cost  of  storing  and  transporting 
material  commodities,  depending  as  it  does  on  their  weight  or  volume; 
while  a  commodity’s  production  cost,  and  its  value  in  use,  depend  on 
other  factors.  - — „ 

It  is  also  worth  recalling  that  to  increase  the  "order"  of  an 
aggregate  is  to  decrease  the  number  of  its  distinct  possible  states. 
This  decreases  the  expected  number  of  symbols  needed  to  identify  a 
state,  and  thus  decreases  the  entropy. 
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